Horse collar data exploration

Binder IPYNB HTML

This notebook presents a systematic movement data exploration workflow. The workflow consists of five main steps:

  1. Establishing an overview by visualizing raw input data records
  2. Putting records in context by exploring information from consecutive movement data records (such as: time between records, speed, and direction)
  3. Extracting trajectories, locations & events by dividing the raw continuous tracks into individual trajectories, locations, and events
  4. Exploring patterns in trajectory and event data by looking at groups of the trajectories or events
  5. Analyzing outliers by looking at potential outliers and how they may challenge preconceived assumptions about the dataset characteristics

The workflow is demonstrated using horse collar tracking data provided by Prof. Lene Fischer (University of Copenhagen) and the Center for Technology & Environment of Guldborgsund Municiplaity in Denmark but should be generic enough to be applied to other tracking datasets.

Raw data import

1. Establishing an overview

The first step in our proposed EDA workflow can be performed directly on raw input data since it does not require temporally ordered data. It is therefore suitable as a first exploratory step when dealing with new data.

Q1.1 Geographic extent: Is the geographical extent as expected and are there holes in the spatial coverage?

The main area (the horste's pasture?) is located south of Nykobing Strandhuse.

However, we find also find two records on the road north west of the main area. Both points have been recorded on 2018-11-14 which is the first day of the dataset.

A potential hypothesis for the origin of these two records is that the horse (or the collar) was transported on 2018-11-14, taking the road from Nykobing Falster south to the pasture.

If we remove these first two records from the dataset, the remainder of the records are located in a small area:

It looks like the horse generally avoids areas without green vegetation since point patterns in these area appear more sparse than in other areas.

Q1.2 Temporal extent: Is the temporal extent as expected and are there holes in the temporal coverage?

On most days there are 48 (+/- 1) records per day. However, there are some days with more records (in Nov 2018 and later between Mai and August 2019).

There is one gap: On 2019-10-18 there are no records in the dataset and the previous day only contains 37 and the following day 27 records.

Q1.3 Spatio-temporal gaps: Does the geographic extent vary over time or do holes appear during certain times?

Considering that the dataset covers a whole year, it may be worthwhile to look at the individual months using small multiples map plots, for example:

The largest change between months seems to be that the southernmost part of the pasture wasn't used in August and September 2019.

2. Putting records in context

The second exploration step puts movement records in their temporal and geographic context. The exploration includes information based on consecutive movement data records, such as time between records (sampling intervals), speed, and direction. Therefore, this step requires temporally ordered data.

Q2.1 Sampling intervals: Is the data sampled at regular or irregular intervals?

For example, tracking data of migratory animals is expected to exhibit seasonal changes. Such changes in vehicle tracking systems however may indicate issues with data collection .

The time delta between consecutive records is usually around 30 minutes.

However, it seems that sometimes the intererval has been decreased to around 15 minutes. This would explain why some days have more than the usual 48 records.

Q2.2 Speed values: Are there any unrealistic movements?

For example: Does the data contain unattainable speeds?

Q2.3 Movement patterns: Are there any patterns in movement direction or speed?

The speed distribution shows no surprising patterns.

There is some variation in movement directions but no directions stand out in the histogram.

Let's look at spatial patterns of direction and speed!

Q2.4 Temporal context: Does the movement make sense in its temporal context?

For example: Do nocturnal animal tracks show movement at night?

The movement speed by hour of day shows a clear pattern throughout the year with earlier and longer fast movements during the summer months and later and slower movements during the winter months.

Temperature context

In addition to time, the dataset also contains temperature information for each record:

Q2.5 Geographic context: Does the movement make sense in its geographic context?

For example: Do vessels follow traffic separation schemes defined in maritime maps? Are there any ship trajectories crossing land?

There are no obvious spatial movement direction patterns.

Low speed records (classes 0.0 and 0.05 m/s) are distributed over the whole area with many points on the outline (fence?) of the area.

Medium speed records (classes 0.1 and 0.15 m/s) seem to be more common along paths and channels.

3. Extracting trajectories & locations / events

The third exploration step looks at individual trajectories. It therefore requires that the continuous tracks are split into individual trajectories. Analysis results depend on how the continuous streams are divided into trajectories, locations, and events.

3.1 Trajectory lines: Do the trajectory lines look plausible or are there indications of out of sequence positions or other unrealistic location jumps?

Due to the 30 minute reporting interval, the trajectories are rather sparse.

The trajectories mostly stay within the (fenced?) area. However, there are a few cases of positions outside the area.

Movement during week #1

3.2 Home/depot locations: Do day trajectories start and end at the same home (for human and animal movement) or depot (for logistics applications) location?

There is no clear preference for a certain home location where the horse would tend to spend the night.

Instead of spliting by date, we can also specify a minimum movement speed and then split the continuous observation when this minimum speed is not reached for a certain time:

3.3 Trajectory length

The length of the daily trajectories varies between 1.6 and 6.2 km. (It is worth noting that this has to be considered a lower bound of the movement due to the sparseness of the tracking data.)

The seasonal trends agree well with the previously discovered seasonal movement speed patterns: winter trajectories tend to be shorter than summer trajectories.

3.4 Covered area

Method 1: Convex hulls around trajectory

Method 2: Buffered trajectory

The ten smallest areas are:

The days with the smallest areas cover areas include the first and the last observation day (since they are only partially recorded). We can remove those:

The smallest area for a complete day was observed on 2018-11-19 with only 1.2 ha:

3.5 Stop detection

Instead of splitting the continuous track into daily trajectories, an alternative approach is to split it at stops. Stops can be defined as parts of the track where the moving object stays within a small area for a certain duration.

Let's have a look at movement of one day and how stop detection parameter settings affect the results:

Let's apply stop detection to the whole dataset:

The spatial distribution reveals preferred stop locations:

Continue exploring MovingPandas

  1. Bird migration analysis
  2. Ship data analysis
  3. Horse collar data exploration
  4. OSM traces